Automate Create Links to Target Module

I can't seem to get my script for linking between a source and target module automatically to work. The biggest hurdle seems to be that I have to be IN the source module in order to execute the script - but I want this to run on a nightly process. I already run a number of such processes successfully in DOORS, but I can't seem to get the script right for this one. 

Here's the template that I'm using. Folders and modules are just placeholders for the actual paths to them.

/* Create_Links_From_Issues_to_Reqs.dxl

This script just hangs until I eventually close out of DOORS. I also tried "hard-coding" the example "LinkByKey.dxl" file, but it still brought up a dialog for me to populate, even though I had plugged in the applicable "choices" in the modified script. I'm guessing that there's something wrong with the way I'm using the Skip, because that's my weak point, usually. What am I missing? Is there a way to run such a script with the modules open in the background instead of launching it from inside the Source module?

Thanks

Re: Automate Create Links to Target Module
PekkaMakinen - Fri Sep 22 02:56:17 EDT 2017

I use a script which indexes almost whole of our DOORS database to create links based on imported Object Identifier data. The index is stored in skips, which are then searched during the linking step to create links. As we have quite a lot of requirements and links to create, this whole operation takes about 3 hours, but at this point we are still running it interactively, not in batch.

I did not really get what and how you are doing, but my script separates the indexing steps and linking steps. First the requirements are indexed to skips and then the skips are used as basis for linking.

BTW, if you are getting some dialogs during the linking, are your module linksets defined as you are linking, direction or link modules?

Re: Automate Create Links to Target Module
O.Wilkop - Fri Sep 22 05:23:57 EDT 2017

it's a bit unclear what you want to do exactly here, I assume this isn't the whole script, but here are some issues:

1. You are using a Skip that gets a string key but you declared it with create() instead of createString(). This can create issues later on when you try to find/retrieve data out of the skiplist, as DOORS might decide to compare the address of the string instead of the contents to determine the correct list entry.

2. You put data into the Skiplist, but you never do anything with it (no retrieving or checking if something with that key exists in the skip), meaning the skiplist is ultimately useless

3. Your script has a bad runtime, as it checks every object in the target module for every object in the source module, it'd be better to run through the target module once, put the interesting attributes into the skiplist, then run through the source module once, check if a corresponding value is in the skiplist and link accordingly.

4. You are using matches() to compare two values, maybe this is intended, but I have a feeling the value in the "txt" variable is not actually a regexp (it might be though, and it should still work if you want to compare exact equality, should that variable contain certain special characters it might be an issue though)

All in all your script should work though, I don't have an issue linking objects inside a batch script. Maybe you can't open the module in edit mode at the time when the script is running or your user doesn't have necessary rights? It could also be that you are opening the module in a filtered view (for o in m loop only loops through visible objects)? All in all I don't see how this script could cause a infinite loop though.

Maybe show us what your two modules look like and based on what you want to link so we can help you further

Re: Automate Create Links to Target Module
Chris Annal - Mon Sep 25 15:13:32 EDT 2017

I overhauled the script to include two Skip Lists in order to compare the attribute values in both modules. It creates links now, but the Source Module attribute can list several "affected requirements" in its attribute. The links are only made for those objects where there is exactly one requirement identified in the "affected requirements" attribute.

Here's the modified code:

I'm not sure how to get this to iterate through the "affected requirements" attribute and create a link for each requirement that is separated by a comma.

Re: Automate Create Links to Target Module
Chris Annal - Mon Nov 06 11:21:20 EST 2017

Actually, I just needed to tweak it a little. Instead of using "==" for comparing the source object attribute value and the target object attribute value, I used "matches". This is the whole script, and it can be run outside the module, assuming the user has the proper access:

Re: Automate Create Links to Target Module
Chris Annal - Tue Nov 07 11:22:07 EST 2017

The problem with my script above is that it "matches" sub-strings in the Target module. That is, if I have an "affected requirement " with the value "ABC-123", it will create links to the requirements "ABC-1", "ABC-12", and "ABC-123". I only want it to create links for EXACT matches. I'll see if I can find some discussions on regular expressions, the "matches" function, and other possible topics here. Thanks for your suggestions!!

Re: Automate Create Links to Target Module
O.Wilkop - Wed Nov 08 05:15:37 EST 2017

If you want to test for equality use "==" instead of matches:

in some cases based on some DXL weirdness you might have to append a empty string to make sure it actually compares the contents and not the place in memory:

Re: Automate Create Links to Target Module
Chris Annal - Wed Nov 08 08:31:58 EST 2017

Thanks for the tip, but that was what I tried before the first time I posted here.

(Rkey >= Jkey) and (Rkey <= Jkey) create links to EVERY object in the Target module.
(Rkey == Jkey) creates only a SINGLE link for the FIRST match, none of the remaining comma-separated list. 

I'll try (Rkey""== Jkey"") and let you know how it goes!

Thanks again!

(Update)

Unfortunately, (Rkey""== Jkey"") behaves the same as the (Rkey == Jkey)

Tried all of the following "if" statements with no luck...

Currently using (matches(Rkey,Jkey)) BUT it creates links to substrings, as well. 

ABC-123 will link to ABC-1,ABC-12, and ABC-123 instead of just ABC-123, for instance.
(other atttempts...)
(Rkey == Jkey) creates links to EVERY object in the Target module.
(Rkey""==Jkey"") creates links to EVERY object in the Target module.
(Rkey >= Jkey) and (Rkey <= Jkey) each create links to EVERY object in the Target module.
((matches(Rkey,Jkey)) && (length(Rkey) == length(Jkey))) creates links to EVERY object in the Target module.
((length (Rkey) == length (Jkey) && matches(Rkey,Jkey))) creates links to EVERY object in the Target module
while (matches(Rkey,Jkey)) freezes DOORS until you hard-exit.
for (matches(Rkey,Jkey)) freezes DOORS until you hard-exit.

I may have to go the ugly route of using an interactive batch file with the popular LinkByKey.dxl script. 

Re: Automate Create Links to Target Module
EHcnck - Tue Nov 21 18:51:50 EST 2017

Skip  skpSourceObjs = createString  // KEY: 'string' Jkey; DATA: 'Object' Handle
Skip  skpTargetObjs = createString  // KEY: 'string' Rkey; DATA: 'Object' Handle

Re: Automate Create Links to Target Module
Chris Annal - Wed Nov 22 09:37:22 EST 2017

Skip  skpSourceObjs = createString  // KEY: 'string' Jkey; DATA: 'Object' Handle
Skip  skpTargetObjs = createString  // KEY: 'string' Rkey; DATA: 'Object' Handle

Thanks EHcnck. I just tried your suggestion, but unfortunately it still allows links to sub-strings. (ABC-123 will link to ABC-1,ABC-12, and ABC-123 instead of just ABC-123, for instance). This is closest to what I want, and I think I just need to work on a better method for comparing the values, I will have to find a way to prevent links being created to target objects with attribute values that are only partial matches for the source object attribute value.

I really thought ((matches(Rkey,Jkey)) && (length(Rkey) == length(Jkey)))  would do the trick when I applied that, but that oddly creates links to EVERY object in the Target module.

Re: Automate Create Links to Target Module
EHcnck - Wed Nov 22 11:15:01 EST 2017

try this.. didn't test it as of yet.

Skip  skpSourceObjs = createString()  // KEY: 'string' Jkey; DATA: 'Object' Handle
Skip  skpTargetObjs = createString ()  // KEY: 'string' Rkey; DATA: 'Object' Handle
Object SourceObj = null
Object TargetObj = null
Module SourceMod = edit("/folder/folder/module", false)
Module TargetMod = read("/folder/folder/module", false)
string LinkMod="/folder/folder/module"
 
// get the affected requirements attribute values for the Source Module objects
pragma encoding, "UTF-8"
pragma runLim, 0

for SourceObj in entire (SourceMod) do
{     
        string src = SourceObj."<source attribute name>"
        if (!null(src)) 
        {
                put(skpSourceObjs, src, SourceObj)
        }
}

for TargetObj in entire (TargetMod) do
{     
        string trg = TargetObj."<target attribute name>"
        if (!null(trg)) 
        {
                put(skpTargetObjs, trg, TargetObj)
        }
}

for SourceObj in skpSourceObjs do
{
        TargetObj = null
        string trg = (string (key skpSourceObjs))
        if (find(skpTargetObjs, trg, TargetObj))
        {
                SourceObj->LinkMod->TargetObj
        }
}
 
delete(skpSourceObjs)
delete(skpTargetObjs)

Re: Automate Create Links to Target Module
Chris Annal - Wed Nov 22 11:38:38 EST 2017

Skip  skpSourceObjs = createString()  // KEY: 'string' Jkey; DATA: 'Object' Handle
Skip  skpTargetObjs = createString ()  // KEY: 'string' Rkey; DATA: 'Object' Handle
Object SourceObj = null
Object TargetObj = null
Module SourceMod = edit("/folder/folder/module", false)
Module TargetMod = read("/folder/folder/module", false)
string LinkMod="/folder/folder/module"
 
// get the affected requirements attribute values for the Source Module objects
pragma encoding, "UTF-8"
pragma runLim, 0

for SourceObj in entire (SourceMod) do
{     
        string src = SourceObj."<source attribute name>"
        if (!null(src)) 
        {
                put(skpSourceObjs, src, SourceObj)
        }
}

for TargetObj in entire (TargetMod) do
{     
        string trg = TargetObj."<target attribute name>"
        if (!null(trg)) 
        {
                put(skpTargetObjs, trg, TargetObj)
        }
}

for SourceObj in skpSourceObjs do
{
        TargetObj = null
        string trg = (string (key skpSourceObjs))
        if (find(skpTargetObjs, trg, TargetObj))
        {
                SourceObj->LinkMod->TargetObj
        }
}
 
delete(skpSourceObjs)
delete(skpTargetObjs)

That's interesting. In the case you suggested, the linking would create links in this fashion:

ABC-123 would link to ABC-123 but...

ABC-1 would link to ABC-1, ABC-12, and ABC-123. 

In other words, kind of the reverse of my original problem. Also, for cases where the source attribute had several target values, separated by commas, it would ignore those cases. (For instance, the source attribute has "ABC-123, ABC-456, ABC-789", the script is probably looking at this as one value).

Re: Automate Create Links to Target Module
EHcnck - Wed Nov 22 13:39:33 EST 2017

code needs testing...

Skip  skpSourceObjs = createString()  // KEY: 'string' Jkey; DATA: 'Object' Handle
Skip  skpTargetObjs = createString ()  // KEY: 'string' Rkey; DATA: 'Object' Handle
Object SourceObj = null
Object TargetObj = null
Module SourceMod = edit("/folder/folder/module", false)
Module TargetMod = read("/folder/folder/module", false)
string LinkMod="/folder/folder/module"
 
// get the affected requirements attribute values for the Source Module objects
pragma encoding, "UTF-8"
pragma runLim, 0

string linkNextValue(string &s) {
    string result = ""
    Regexp hasField = regexp "[ ,;\t\n]*([^,;\t\n]+)[ ,;\t\n]*"

    while ( (null result) and (! null s) ) {
        hasField s
        result = s[match 1]
        s = s[end(0)+1:]
    }

        delete hasField
        trim_(result)
    return result
}//End linkNextValue

for SourceObj in entire (SourceMod) do
{
        if (null(TargetObj)) continue
        if (isDeleted(TargetObj)) continue
        
        string src = SourceObj."<source attribute name>"
        if (!null(src)) 
        {
                put(skpSourceObjs, src, SourceObj)
        }
}

for TargetObj in entire (TargetMod) do
{
        if (null(TargetObj)) continue
        if (isDeleted(TargetObj)) continue
        
        string trg = TargetObj."<target attribute name>"
        if (!null(trg)) 
        {
                put(skpTargetObjs, trg, TargetObj)
        }
}

for SourceObj in skpSourceObjs do
{
    TargetObj = null
    string trgValues = (string (key skpSourceObjs))
        string value = null
        
        while ( !null trgValues ) {
                value = linkNextValue(trgValues)
                
                if (!null(value)) {
                        if (find(skpTargetObjs, value, TargetObj)) {
                                SourceObj->LinkMod->TargetObj
                        } else {
                                //error numKeysNotFound
                        }
                } else {
            //error numKeysNotFound
                }
        }
}
 
delete(skpSourceObjs)
delete(skpTargetObjs)

Re: Automate Create Links to Target Module
Chris Annal - Mon Nov 27 10:04:54 EST 2017

Skip  skpSourceObjs = createString()  // KEY: 'string' Jkey; DATA: 'Object' Handle
Skip  skpTargetObjs = createString ()  // KEY: 'string' Rkey; DATA: 'Object' Handle
Object SourceObj = null
Object TargetObj = null
Module SourceMod = edit("/folder/folder/module", false)
Module TargetMod = read("/folder/folder/module", false)
string LinkMod="/folder/folder/module"
 
// get the affected requirements attribute values for the Source Module objects
pragma encoding, "UTF-8"
pragma runLim, 0

string linkNextValue(string &s) {
    string result = ""
    Regexp hasField = regexp "[ ,;\t\n]*([^,;\t\n]+)[ ,;\t\n]*"

    while ( (null result) and (! null s) ) {
        hasField s
        result = s[match 1]
        s = s[end(0)+1:]
    }

        delete hasField
        trim_(result)
    return result
}//End linkNextValue

for SourceObj in entire (SourceMod) do
{
        if (null(TargetObj)) continue
        if (isDeleted(TargetObj)) continue
        
        string src = SourceObj."<source attribute name>"
        if (!null(src)) 
        {
                put(skpSourceObjs, src, SourceObj)
        }
}

for TargetObj in entire (TargetMod) do
{
        if (null(TargetObj)) continue
        if (isDeleted(TargetObj)) continue
        
        string trg = TargetObj."<target attribute name>"
        if (!null(trg)) 
        {
                put(skpTargetObjs, trg, TargetObj)
        }
}

for SourceObj in skpSourceObjs do
{
    TargetObj = null
    string trgValues = (string (key skpSourceObjs))
        string value = null
        
        while ( !null trgValues ) {
                value = linkNextValue(trgValues)
                
                if (!null(value)) {
                        if (find(skpTargetObjs, value, TargetObj)) {
                                SourceObj->LinkMod->TargetObj
                        } else {
                                //error numKeysNotFound
                        }
                } else {
            //error numKeysNotFound
                }
        }
}
 
delete(skpSourceObjs)
delete(skpTargetObjs)

There seems to be some problems with the trim function in various versions of DOORS, per the discussion in this forum here:
https://www.ibm.com/developerworks/community/forums/html/topic?id=efdabdc1-deab-4c63-acb2-037677a3ceeb&ps=25

I found a separate forum page where a user had created their own "trim" function, so I added that to your suggested script (at the top of the script, before linkNextValue). Again, no joy. The script ran, but it didn't create any links. Just for reference, here's the "trim" function that I added:

string rightTrim(string s) {
    if (null(s)) return "";
    int first = 0;
    int last = length(s) - 1;
    while( (last > -1) && (isspace(s[last])) ){
       last--;
    }
    return(s[first:last]);
}
 
string leftTrim(string s) {
    if (null(s)) return "";
    int first = 0;
    int last = length(s) - 1;
    while( (isspace(s[first])) && (first < last) ) {
       first++;
    }
    return(s[first:last]);
}
 
 
string trim_(string s) {
    return(leftTrim(rightTrim(s)));
}

This was found at this forum site: https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014779255&ps=25. I only went with this function because I was getting errors from the trim_ function if I didn't write my own...

-E- DXL: <Line:25> incorrectly concatenated tokens
-E- DXL: <Line:25> undeclared variable (trim_)
-I- DXL: All done. Errors reported: 2. Warnings reported: 0.

Re: Automate Create Links to Target Module
EHcnck - Mon Nov 27 10:11:26 EST 2017

can you provide sample modules? what version of DOORS are you running?

Re: Automate Create Links to Target Module
Chris Annal - Mon Nov 27 12:26:54 EST 2017

Thanks for looking into this as much as you have. I really appreciate it. I archived an "Example" project that includes the source, target and link modules I was testing this with. I've tested this in DOORS 9.3.0.5, 9.3.07, and 9.3.0.11. Per the other discussions in the forum, these versions should be able to work with the "trim_" function, but I get the errors mentioned above. 

I've also attached the script that I updated with my home-made TRIM function. I don't get any errors with this script, but it doesn't create any links, either.

Thanks again!

Re: Automate Create Links to Target Module
EHcnck - Mon Nov 27 14:29:08 EST 2017

this version seems to work for me using 9.6.1.8/9

Re: Automate Create Links to Target Module
Chris Annal - Mon Nov 27 14:43:13 EST 2017

By George, I think you've done it!! Those final tweaks seem to have it working for me, too! Thanks so much for the time and effort you put in to making this work!

: